Preparing Korean Data for the Shared Task on Parsing Morphologically Rich Languages

This document gives a brief description of Korean data prepared for the SPMRL 2013 shared task (Seddah et al., 2013). A total of 27,363 sentences with 350,090 tokens are used for the shared task. All constituent trees are collected from the KAIST Treebank and transformed to the Penn Treebank style. All dependency trees are converted from the transformed constituent trees using heuristics and labeling rules designed specifically for the KAIST Treebank. In addition to the gold-standard morphological analysis provided by the KAIST Treebank, two sets of automatic morphological analysis are provided for the shared task, one is generated by the HanNanum morphological analyzer, and the other is generated by the Sejong morphological analyzer. 1 Constituent Treebank All constituent trees are collected from the KAIST Treebank (Choi et al., 1994). The KAIST Treebank contains about 31K manually annotated constituent trees from 97 different sources (e.g., newspapers, novels, textbooks). After filtering out trees with annotation errors, a total of 27,363 trees with 350,090 tokens are collected. Table 1 shows distributions of the training, development, and evaluation sets used for the shared task. Train Develop Evaluate Trees 23,010 2,066 2,287 Tokens 296,446 25,278 28,366 Table 1: Distributions of the training, development, and evaluation sets used for the shared task. Constituent trees in the KAIST Treebank also come with manually inspected morphological analysis based on ‘eojeol’. An eojeol contains root-forms of word tokens agglutinated with grammatical affixes (e.g., case particles, ending markers). An eojeol can consist of more than one word token; for instance, a compound noun “bus stop” is often represented as one eojeol in Korean,!Q€o&Ò⌦¿”Å©ú, which can be broken into two word tokens,!Q€o (bus) and&Ò⌦ ¿”Å©ú (stop). Each eojeol in the KAIST Treebank is separated by white spaces regardless of punctuation. Figure 1 shows morphological analysis for a sentence, “I drank cognac.” in Korean. (Cognac) . I cognac drank + +(+Cognac+)+ + + +. I+tpc cognac+(+Cognac+)+obj drink+past+final+. Figure 1: Morphological analysis for a sentence, “I drank cognac.” in Korean, where each morpheme is separated by a plus sign (+). tpc: topical auxiliary, obj: objective case particle, past: past-tense ending marker, final: final ending marker.